On Bayesian principal component analysis

A complete Bayesian framework for Principal Component Analysis (PCA) is proposed in this paper. Previous model-based approaches to PCA were usually based on a factor analysis model with isotropic Gaussian noise. This model does not impose orthogonality constraints, contrary to PCA. In this paper, we propose a new model with orthogonality restrictions, and develop its approximate Bayesian solution using the variational approximation and results from directional statistics. The Bayesian solution provides two notable results in relation to PCA. The first are uncertainty bounds on principal components (PCs), and the second is an explicit distribution on the number of relevant PCs. The posterior distribution for the PCs is found to be of the von-Mises-Fisher type. This distribution—and its associated hypergeometric function, 0F1—is studied. Numerical reductions are revealed, leading to a stable and efficient Orthogonal Variational PCA (OVPCA) algorithm. OVPCA provides the inferences sought above. Its performance is illustrated in simulation, and for a sequence of medical scintigraphic images.